{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "ee2a3a21",
   "metadata": {},
   "source": [
    "# QA Generation\n",
    "This notebook shows how to use the `QAGenerationChain` to come up with question-answer pairs over a specific document.\n",
    "This is important because often times you may not have data to evaluate your question-answer system over, so this is a cheap and lightweight way to generate it!\n",
    "\n",
    "\n",
    "And You can find the origin notebook in [LangChain example](https://github.com/hwchase17/langchain/blob/master/docs/use_cases/evaluation/qa_generation.ipynb), and this example will show you how to set the LLM with GPTCache so that you can cache the data with LLM. You can also try this example on [Google Colab](https://colab.research.google.com/drive/1avQR-ANehLmyfDcKR_2A_IycxwCpTa-k?usp=sharing)."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bb3f83a2",
   "metadata": {},
   "source": [
    "## Go into GPTCache\n",
    "\n",
    "Please [install gptcache](https://gptcache.readthedocs.io/en/latest/index.html#) first, then we can initialize the cache.There are two ways to initialize the cache, the first is to use the map cache (exact match cache) and the second is to use the DataBse cache (similar search cache), it is more recommended to use the second one, but you have to install the related requirements.\n",
    "\n",
    "Before running the example, make sure the `OPENAI_API_KEY` environment variable is set by executing `echo $OPENAI_API_KEY`. If it is not already set, it can be set by using `export OPENAI_API_KEY=YOUR_API_KEY` on Unix/Linux/MacOS systems or `set OPENAI_API_KEY=YOUR_API_KEY` on Windows systems. And there is `get_msg_func` for the cache settings:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "29ea2668",
   "metadata": {},
   "outputs": [],
   "source": [
    "# get the content(only question) form the prompt to cache\n",
    "def get_msg_func(data, **_):\n",
    "    return data.get(\"messages\")[-1].content"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ae0efe00",
   "metadata": {},
   "source": [
    "### 1. Init for exact match cache"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "7892a80f",
   "metadata": {},
   "outputs": [],
   "source": [
    "# from gptcache import cache\n",
    "# cache.init(pre_embedding_func=get_msg_func)\n",
    "# cache.set_openai_key()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7cc2c7a7",
   "metadata": {},
   "source": [
    "### 2. Init for similar match cache\n",
    "\n",
    "When initializing gptcahe, the following four parameters are configured:\n",
    "\n",
    "- `pre_embedding_func`: pre-processing before extracting feature vectors, it will use the `get_msg_func` method\n",
    "- `embedding_func`: the method to extract the text feature vector\n",
    "- `data_manager`: DataManager for cache management\n",
    "- `similarity_evaluation`: the evaluation method after the cache hit\n",
    "\n",
    "The `data_manager` is used to audio feature vector, response text in the example, it takes [Milvus](https://milvus.io/docs) (please make sure it is started), you can also configure other vector storage, refer to [VectorBase API](https://gptcache.readthedocs.io/en/latest/references/manager.html#module-gptcache.manager.vector_data)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "c00c0422",
   "metadata": {},
   "outputs": [],
   "source": [
    "from gptcache import cache\n",
    "from gptcache.embedding import Onnx\n",
    "from gptcache.manager import CacheBase, VectorBase, get_data_manager\n",
    "from gptcache.similarity_evaluation.distance import SearchDistanceEvaluation\n",
    "\n",
    "\n",
    "onnx = Onnx()\n",
    "cache_base = CacheBase('sqlite')\n",
    "vector_base = VectorBase('milvus', host='127.0.0.1', port='19530', dimension=onnx.dimension)\n",
    "data_manager = get_data_manager(cache_base, vector_base)\n",
    "cache.init(\n",
    "    pre_embedding_func=get_msg_func,\n",
    "    embedding_func=onnx.to_embeddings,\n",
    "    data_manager=data_manager,\n",
    "    similarity_evaluation=SearchDistanceEvaluation(),\n",
    "    )\n",
    "cache.set_openai_key()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "083c33dd",
   "metadata": {},
   "source": [
    "After initializing the cache, you can use the LangChain Chat Models with `gptcache.adapter.langchain_models`. At this point **gptcache** will cache the answer, the only difference from the original example is to change `chat=ChatOpenAI(temperature=0)` to `chat = LangChainChat(chat=ChatOpenAI(temperature=0))`, which will be commented in the code block.\n",
    "\n",
    "Then you will find that it will be more fast when search the similar content, let's play with it."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8a0b2c33",
   "metadata": {},
   "source": [
    "## Getting Started"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "33d3f0b4",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain.document_loaders import TextLoader\n",
    "from langchain.text_splitter import RecursiveCharacterTextSplitter, TextSplitter\n",
    "\n",
    "text_splitter =  RecursiveCharacterTextSplitter(chunk_overlap=500, chunk_size=2000)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "2029a29c",
   "metadata": {},
   "outputs": [],
   "source": [
    "loader = TextLoader(\"./state_of_the_union.txt\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "87edb84c",
   "metadata": {},
   "outputs": [],
   "source": [
    "doc = loader.load()[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "04125b6d",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain.chat_models import ChatOpenAI\n",
    "from langchain.chains import QAGenerationChain\n",
    "from gptcache.adapter.langchain_models import LangChainChat\n",
    "\n",
    "# chat = ChatOpenAI(temperature=0) # using the following code to cache with gptcache\n",
    "chat = LangChainChat(chat=ChatOpenAI(temperature=0))\n",
    "\n",
    "chain = QAGenerationChain.from_llm(chat, text_splitter=text_splitter)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "4f1593e4",
   "metadata": {},
   "outputs": [],
   "source": [
    "qa = chain.run(doc.page_content)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "ee831f92",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'question': 'What did President Zelenskyy say in his speech to the European Parliament?',\n",
       " 'answer': \"President Zelenskyy said in his speech to the European Parliament that 'Light will win over darkness.'\"}"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "qa[1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7028754e",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
